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*A  small,  self-contained  microprocessor  displays  a  manikin  on  a  TV  screen  in 
various  orientations.  Subjects  are  required  to  choose  lateralities  depending 
on  a  shape  also  presented  on  the  display,  and  performance  (accuracy  and  reac¬ 
tion  time)  is  automatically  recorded. 

This  task  has  been  evaluated  at  the  USAF  School  of  Aerospace  Medicine.  Data 
are  presented  to  show  that  the  task  is  simple  to  use,  can  be  learned  by  all 
subjects,  and  that  plateau  performance  can  be  quickly  established.^ 
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measures  of  performance  were  compared;  and  repetitive  effects  were  sought  with 
similarities  of  presentation  in  sequence. 

The  analyses  showed  that  control  performance  with  this  task  was  essentially 
constant  over  the  experimental  period. 
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EVALUATION  OF  A  MANIKIN  PSYCHOMOTOR  TASK 


INTRODUCTION 

Reaction  time  as  a  measure  of  human  performance  has  been  in  use  for  over 
100  years.  Response  to  a  single  stimulus  is  referred  to  as  simple  reaction 
time,  while  the  multiple  stimulus  situation  is  known  as  choice  reaction  time. 
Preference  for  reaction  time  as  a  measure  of  human  performance  has  been  predi¬ 
cated  on  the  proposition  that  behavior  takes  time  and  the  length  of  time  re¬ 
corded  as  reaction  time  is  an  accurate  appraisal  of  at  least  some  segment  of 
human  behavior.  Since  reaction  time  is  commonly  defined  as  the  time  from  the 
onset  of  a  stimulus  to  the  initiation  of  the  subject's  response,  cognitive 
rather  than  motor .v  processes  are  reflected  in  reaction  time.  To  exploit  this 
measurement  fully,  more  complex  stimulus  arrays  are  necessary.  For  as  William 
James  (3)  noted,  it  is  only  when  the  experiments  are  complicated  that  there  is 
a  chance  for  anything  like  an  intellectual  operation  to  occur. 

Successful  performance  in  modern  military  systems  is,  to  a  large  extent, 
less  dependent  on  skillful  physical  manipulation  of  controls,  than  it  is  on 
careful  planning  of  resource  management  and  supervision  of  automated  or  semi- 
automated  system  operation.  At  the  same  time  highly  trained  and  skilled  oper¬ 
ators  are  typically  faced  with  complex  stimulus  arrays  which  nonetheless 
require  discrete  control  actions.  An  experimenter  who  seeks  to  tap  similar 
psychomotor  behavior  in  the  laboratory  with  unskilled  or  minimally  trained 
subjects  faces  a  difficult  decision  in  task  selection.  For  these  reasons  more 
cognitively  based  performance  measures  that  are  not  system  dependent  have 
become  increasingly  important  for  laboratory  performance  measurement.  The 
manikin  task  was  developed  at  the  Royal  Air  Force  (RAF)  Institute  of  Aviation 
Medicine,  Farnborough,  United  Kingdom,  as  a  complex  reaction  time  task  that 
related  to  performance  in  modern  systems  without  being  identified  with  any 
particular  system. 

For  the  manikin  task  (or  any  task)  to  be  useful  as  a  laboratory  analog  of 
some  component  of  real-world  system  performance,  it  must  meet  several  practi¬ 
cal  criteria.  Since  time  is  generally  limited  for  training  of  subjects, 
acquisition  of  asymptotic  performance  should  not  require  protracted  training. 
Also,  it  is  impractical  for  all  subjects  to  receive  training  on  precisely  the 
same  schedule.  Therefore,  asymptotic  performance  should  not  be  training- 
schedule  dependent.  If  preasymptotic  performance  is  to  be  useful  as  a 
measure,  then  the  relative  positions  of  individuals'  performance  should  be 
constant  across  some  period  of  training  and  differential  task  stability  (reli¬ 
ability)  should  be  evident.  For  the  task  to  have  validity,  it  should  be  shown 
to  be  sensitive  to  manipulations  of  interest;  e.g.,  the  manikin  test  has  been 
shown  to  be  sensitive  to  mild  hypoxia. 

Although  the  validity  of  the  manikin  task  appeared  well-established  and 
had  been  used  to  measure  performance  during  hypoxia  (2),  reliability  had  not 
been  systematical ly  investigated.  Additional  factors  influencing  acquisition 
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of  asymptotic  performance  also  remained  unidentified.  For  these  reasons  data 
collection  was  instituted  to  verify  the  utility  of  the  manikin  task.  Addi¬ 
tional  analyses  were  undertaken  to  determine  intratask  variables  that  related 
to  subject  performance.  Subjective  impressions  of  performance  were  compared 
with  actual  measures  of  performance  in  a  later  phase  of  the  experiment.  These 
data  were  also  analyzed  to  seek  repetition  effects. 


Format  of  Task 

The  apparatus  consists  of  a  microprocessor,  power  supply  box,  TV  monitor, 
and  two  hand-held  buttons  (Fig.  1).  A  digital  recorder  is  also  required. 

The  manikin  task  presents  a  video  picture  of  a  little  man  (manikin)  in 
outline  (Fig.  2).  The  arms  are  outstretched  and  hold  a  square  in  the  right 
hand  and  a  disc  in  the  left  hand,  or  vice  versa.  The  manikin  may  be  either 
upright  or  inverted  and,  in  addition,  may  face  towards  or  away  from  the  sub¬ 
ject  (four  types  of  presentations).  Facial  and  clothing  details  are  visible 
in  the  facing  presentation.  Below  the  manikin  either  a  disc  or  square  is 
shown.  The  subjects'  task  is  to  determine  in  which  hand  the  manikin  holds  the 
identical  shape  to  that  presented  below  it  and,  having  decided,  to  press  the 
button  corresponding  to  that  hand. 

The  manikin  picture  is  generated  in  a  small  microcomputer  which  also  con¬ 
trols  the  presentation  timing.  Each  manikin  is  presented  for  2  seconds  and 
occluded  for  1  second.  In  addition,  the  manikin  is  occluded  following  each 
response.  A  series  of  96  manikins  is  presented  in  sequence.  The  sequence  is 
fixed,  but  it  always  starts  at  a  different  place.  The  96  presentations  con¬ 
sist  of  6  segments  of  16  presentations,  and  each  segment  contains  all  the  16 
possibilities  of  presentation  but  in  a  different  random  order.  Thus,  for  the 
purpose  of  analysis,  each  segment  of  16  (48  seconds)  is  of  equal  difficulty. 
The  96  manikin  presentations  make  a  period  of  4.8  minutes.  Four  periods 
interspersed  with  3  rest  periods  of  2  minutes  are  grouped  to  make  a  session  of 
25.2  minutes.  The  experimenter  presses  the  "prime"  button  on  the  microcom¬ 
puter  to  start  a  session  of  testing.  Before  each  period  the  subject  is  warned 
by  a  tone  for  4  seconds  before  the  first  manikin  presentation  of  that  period. 
At  the  end  of  the  last  period,  a  warbling  tone  informs  that  the  session  is 
ended  and  that  the  microprocessor  timer  requires  restarting  by  pressing  the 
"reset"  button.  Pressing  the  reset  button  before  the  end  of  the  session 
resets  the  timer,  and  the  prime  button  restarts  it. 

The  microcomputer  also  generates  a  recording  signal  of  the  psychomotor 
data.  When  linked  with  a  digital  recorder,  the  following  data  are  recorded 
after  each  presentation:  a  number  from  0-16  denoting  the  type  of  presenta¬ 
tion;  after  a  space,  a  1,  0,  or  *  denoting  whether  a  correct,  incorrect,  or  no 
response  was  recorded;  and  after  a  further  space,  the  reaction  time  in  seconds 
to  2  decimal  places.  At  the  end  of  the  session,  a  surmary  statement  of  total 
numbers  of  correct,  incorrect,  and  no  responses  and  the  aggregate  of  reaction 
time  is  recorded. 

The  logic  of  the  microprocessor  is  fixed;  neither  the  timing  nor  presen¬ 
tation  sequence  can  be  altered. 
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The  manikin  task  data  were  recorded  on  the  disc  storage  of  a  PDP  11/34 
computer.  An  ASR  40  line  printer  was  used  in  parallel  to  the  computer  to 
record  a  hard  copy  of  the  data.  A  small  switch  box  was  employed  whereby  indi¬ 
ces  containing  subject  and  run  details  could  be  inserted  into  the  disc  storage 
by  the  line  printer. 


Experimental  Protocol 

Two  experiments  were  performed  using  the  manikin  task.  The  first  sought 
to  examine  the  variables  that  govern  acquisition  of  plateau  performance,  i.e., 
those  that  control  the  shape  of  the  learning  curve.  Eighteen  naive  subjects 
were  exposed  to  10  sessions  of  training.  They  were  divided  into  the  following 
groupings:  (a)  by  training  frequency--6  subjects  trained  twice  a  day,  6  sub¬ 
jects  trained  once  a  day,  and  6  subjects  trained  every  alternate  working  day; 
(b)  by  age--below  28  years  (7),  between  28  and  38  years  (7),  and  over  38  years 
(4);  and  (c)  by  occupational  level--viz,  professional  scientists  (6),  noncom¬ 
missioned  officers  (6),  and  airmen  (6).  Subjects  were  not  permitted  to  exam¬ 
ine  the  data  during  the  study. 

The  second  experiment  examined  the  relationship  of  subjective  and  objec¬ 
tive  measures  of  performance.  Subjective  assessment  of  performance  has  been 
used  extensively  where  direct  objective  measures  of  performance  are  denied. 
The  same  18  subjects  were  exposed  to  five  further  sessions  with  the  manikin 
task,  one  per  day,  on  consecutive  days.  They  were  asked  to  estimate  both  the 
workload  effort  and  their  performance  against  seven-point  scales  (Figs.  3  and 
4).  This  experiment  also  examined  whether  the  shape  (disc  or  square)  or  side 
(right  or  left)  affected  overall  performance  and  also  checked  the  difficulty 
of  each  of  the  four  types  of  presentations  for  the  subjects. 

Repetition  effects  were  also  considered  using  data  from  the  second  exper¬ 
iment.  The  manikin  task  sequence  is  fixed;  thus  all  subjects  were  presented 
with  identical  sequences  (but  different  start  positions).  The  similarity  of 
presentations  to  the  immediate  preceding  presentation  varies  as  shapes,  sides, 
upright  or  inverted,  forward  and  away  facing,  are  altered.  The  sequences  per¬ 
mitted  7  levels  of  similarity  to  be  compared  from  exactly  comparable  to  gross 
dissimilarity  (every  variable  reversed).  The  data  from  the  sessions  conducted 
during  the  second  experiment  (subjective  versus  objective  assessment)  were 
analyzed  to  detect  repetition  effects,  i.e.,  enhanced  performance  with  simi¬ 
larity  of  presentations  in  sequence.  Finally,  an  analysis  was  performed  on 
the  stability  or  reliability  of  the  task  data  over  the  entire  period  of  the 
experiment . 


Experimental  Procedure 

Subjects  sat  in  a  darkened,  quiet  room,  viewed  a  7-inch  (17.78  cm)  video 
display  from  approximately  38  inches  (96.52  cm)  (approximately  10°  visual 
angle)  and  held  press-button  boxes  in  either  hand.  Only  one  session  of  per¬ 
formance  (25.2  min)  was  conducted  for  each  subject  at  a  time.  In  some  in¬ 
stances,  ear  defenders  were  used  when  external  noise  could  not  be  controlled. 
An  experimental  monitor  briefed  the  subjects  and  controlled  the  apparatus  and 
computer.  Distraction  to  the  subjects  was  minimized  at  all  times. 
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INSTRUCTIONS-  Circle  the  number  that  best  describes  the  workload  you  experi¬ 
enced  while  performing  the  task  during  this  session. 


1.  NOTHING  TO  DO;  NO  SYSTEM  DEMANDS 

2.  LITTLE  TO  DO;  MINIMUM  SYSTEM  DEMANDS 

3.  ACTIVE  INVOLVEMENT  REQUIRED:  EASY  TO  KEEP  UP 

4.  CHALLENGING,  BUT  MANAGEABLE 

5.  EXTREMELY  BUSY;  BARELY  ABLE  TO  KEEP  UP 

6.  OVERLOADED;  HIGH  CHANCE  OF  ERROR 

7.  UNMANAGEABLE;  UNACCEPTABLE 

Figure  3.  Workload  estimate. 


INSTRUCTIONS:  Circle  the  number  of  the  appropriate  term.  Compared  to  my  per¬ 
formance  last  session,  my  performance  on  this  session  was . 


1.  MUCH  WORSE 

2.  WORSE 

3.  SLIGHTLY  WORSE 

4.  SAME 

5.  SLIGHTLY  BETTER 

6.  BETTER 

7.  MUCH  BETTER 


Figure  4.  Performance  estimate. 
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The  data  on  disc  storage  was  transferred  to  an  IBM  360  computer  where  all 
analyses  were  performed  using  the  Statistical  Analysis  System  (SAS). 


RESULTS 

Experiment  1  examined  the  acquisition  of  plateau  performance.  For  the 
purpose  of  the  experiment,  plateau  performance  was  defined  as  not  exceeding 
±  5%  of  the  mean  reaction  time  (RT)  of  the  previous  2  sessions.  The  number  of 
sessions  required  to  achieve  this  criterion  are  given  in  Table  1.  The  data 
are  classified  by  occupational  level.  In  one  case,  a  strategy  adopted  by  one 
subject  (JB)  allowed  him  to  improve  his  performance  throughout  the  experi¬ 
ment.  Therefore,  he  did  not  reach  the  criterion  set  and  was  excluded  from  the 
analysis.  Table  2  classifies  the  data  by  frequency  of  training.  Table  3 
classifies  the  number  of  sessions  to  plateau  by  age.  Both  a  one-way  analysis 
of  variance  and  a  nonparametric  H  test  on  ranked  data  failed  to  detect  any 
significant  differences  in  number  of  sessions  to  plateau  between  any  of  the 
above  three  groupings  (training,  occupational  level,  and  age)  at  the  0.1 
level  . 

A  repeated  measurements  analysis  of  variance  procedure  was  used  to  ana¬ 
lyze  the  reaction  time  (RT)  data  for  training  group,  session,  and  period  dif¬ 
ferences,  plus  their  interactions.  (See  Table  4  for  the  sources  of  variation 
in  the  analysis.)  There  was  strong  evidence  ( p< .  001 )  that  the  pattern  of  the 
period  means  was  not  consistent  from  one  session  to  another  (period  by  session 
interaction).  This  inconsistency  is  a  reflection  of  more  improvement  over 
periods  for  the  initial  sessions  than  for  the  latter  sessions,  a  finding  which 
was  expected.  The  analysis  failed  to  detect  any  differences  among  the  three 
training  groups  in  this  improvement  in  performance.  That  is,  the  training 
group  did  not  show  any  significant  interaction  with  period  or  session.  Of 
course,  the  overall  performance  improved  over  the  ten  sessions  ( p<  .001 )  . 

There  was  some  indication  of  an  overall  difference  among  the  three  train¬ 
ing  groups  (p<.10).  (See  Table  5  for  means.)  The  group  that  trained  on 
alternate  days  had  a  lower  mean  reaction  time  than  that  of  the  other  two 
groups.  The  repeated  measurements  analysis  procedure  was  also  used  to  a  yze 
the  "number  of  correct  responses"  and  the  mean  reaction  time  for  the  c  rect 
responses.  The  testing  results  were  very  comparable  among  the  analyses  for 
the  three  measurements .  Again  the  group  that  trained  on  alternate  days  had  a 
better  accuracy  mean  (more  correct  responses)  and  lower  mean  reaction  ;  m  for 
the  correct  responses  only.  It  must  be  remembered  that  these  are  /'rail 
training  group  mean  differences  (averaged  over  all  sessions)  and  the  .ore  do 
not  reflect  a  training  effect. 

The  associated  mean  reaction  time,  etc.,  are  also  given  in  Table  b  for 
the  age  and  occupational  level  groupings.  Generally,  no  significant  differ¬ 
ences  were  detected  among  these  groupings.  The  analyses  on  each  of  the  three 
groupings  (training,  occupational  level,  and  age)  were  performed  ignoring  the 
other  two  groupings. 

Further  analyses  of  the  data  showed  that  the  manikin  presentations  insed 
varying  degrees  of  difficulty.  Table  6  lists  the  mean  reaction  time  for  the 
three  groups  divided  by  presentation  type. 
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TABLE  1. 


Group _ 

Professional 
scienti st 


NCOS 


Airmen 


NUMBER  OF  SESSIONS  TO  CRITERION  (PLATEAU  PERFORMANCE) 
(CLASSIFIED  BY  OCCUPATIONAL  LEVEL) 


Subject 


Number 


Mean 


Min 


Max 


WS 

SG 

DB 

LP 

PD 

DM 


4 

4 

10 

5 
8 
5 


10 


BW 

PP 

EC 

FA 

DT 

JB 


5 

7 

7 

5 

5 

★ 


(Did 


not  reach 
5.8 


pi  ateau) 
5 


RC 

TB 

VP 

TW 

JS 

MF 


6 

7 

9 

5 

5 

7 


6.5 


10 


TABLE  2.  NUMBER  OF  SESSIONS  TO  CRITERION  (PLATEAU  PERFORMANCE) 
(CLASSIFIED  BY  FREQUENCY  OF  TRAINING) 


Group _ 

Twice  daily 


Once  daily 


A1  ternate 
days 


Subject _ Number _  Mean _ Min _ Max 


PD  8 

DM  5 

FA  5 

DT  5 

JS  5 

MF  7 


5.8  5  8 


DB 

LP 

PP 

EC 

VP 

TW 


10 

5 

7 

7 

9 

5 

7.1  5  10 


WS 

SG 

BW 

JB 

RC 

TB 


4 


5 

★ 

6 
7 


(Did  not  reach  plateau) 
5.2  4 


7 


11 


TABLE  3.  NUMBER  OF  SESSIONS  TO  CRITERION  (PLATEAU  PERFORMANCE) 

(CLASSIFIED  BY  AGE) 


<28  JB 

DT 
RC 
TB 
VP 
TW 
ME 


*  (Did  not  reach  plateau) 

5 

6 
7 
9 
5 
7 


9 


12 


TABLE  4.  SOURCES  OF  VARIATION  FOR  THE  REPEATED  MEASUREMENTS  ANALYSIS 


Oegrees  of 

Source  freedom 


Between  training  groups  2 
Between  subjects*  within  training  groups  14 

Between  sessions  9 
Interaction  of  training  groups  and  sessions  18 
Interaction  of  subjects  and  sessions  within  groups  134 

Between  periods  3 
Interaction  of  training  groups  and  periods  6 
Interaction  of  subjects  and  periods  within  groups  44 

Interaction  of  sessions  and  periods  27 
Interaction  of  training  groups  and  sessions  and  periods  54 
Interaction  of  subjects  and  sessions  and  periods  within 

groups  404 


♦Data  for  JB  were  not  included  in  the  analysis. 


13 


TABLE  5.  PERFORMANCE  IN  THE  THREE  GROUPS 


Mean  RT  (sec)  for 

Group _ ERT  (sec) _ No.  correct _ correct  responses 


Training  frequency 


Twice  daily 

78.18 

90.8 

.809 

Once  daily 

88.22 

91.4 

.912 

Alternate  days 

68.24 

93.1 

.709 

Occupational  level 


Professional 


scientist 

83.38 

91.5 

.865 

NCOs 

79.37 

91.1 

.820 

Airmen 

71.08 

92.7 

.738 

Age 


<28 

70.57 

92.6 

.732 

28-38 

79.51 

92.1 

.822 

>38 

87.15 

89.9 

.?Jl 

14 


TABLE  6.  PERFORMANCE  AND  PRESENTATION  TYPE 


Group 

Mean 

reaction  time 

(sec)  for  correct 

responses 

Erect 

away 

Erect 

facing 

Inverted 

facing 

Inverted 

away 

Training  frequency 

Twice  daily 

.741 

.803 

.830 

.864 

Once  daily 

.845 

.921 

.929 

.956 

Alternate  days 

.643 

.728 

.705 

.762 

Total 

.793 

.817 

.821 

.860 

Occupational  level 

Professional 

scienti st 

.799 

.878 

.869 

.907 

NCOs 

.729 

.798 

.866 

.896 

A  i  rinen 

.694 

.769 

.720 

.772 

Age 

<28 

.684 

.750 

.727 

.771 

28-38 

.740 

.815 

.846 

.892 

>38 

.826 

.913 

.923 

.94  7 
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A  repeated  measurements  analysis  was  performed  for  each  presentation  type 
separately  using  only  the  RT  values  for  which  the  associated  responses  were 
correct  in  order  to  eliminate  the  reaction  times  with  the  incorrect  re¬ 
sponses.  The  results  for  each  analysis  were  very  comparable  to  the  previous 
results;  i.e.,  no  additional  meaningful  information  was  gained.  Generally, 
the  erect  away  presentation  had  the  lowest  mean  reaction  time  with  the  erect 
facing,  inverted  facing,  and  inverted  away  groups  having  progressively  in¬ 
creasing  reaction  times.  The  means  are  also  given  in  Table  6  for  the  occupa¬ 
tional  level  and  age  groupings.  No  statistical  tests  were  performed  on  these 
groupings. 

The  second  experiment  examined  the  correlation  of  subjective  and  objec¬ 
tive  measures  of  performance.  After  each  session  the  subjects  reported  on  a 
scale  from  1-7  their  impressions  of  workload  and  performance.  These  data  were 
compared  with  the  performance  data  (RT)  from  the  manikin  task.  As  subject  JB 
used  a  different  strategy  for  the  task  than  all  the  other  subjects,  his  data 
were  omitted  from  the  analysis.  Table  7  lists  the  Pearson  product -moment  cor¬ 
relation  coefficients  between  the  mean  RT  data  (for  all  sessions)  and  each  of 
these  two  subjective  values  among  the  17  subjects.  These  values  are  low  and 
do  not  differ  statistically  from  zero.  Table  8  lists  the  correlation  coeffi¬ 
cient  for  performance  and  RT  as  well  as  workload  and  RT  among  the  5  sessions 
within  each  subject  separately.  Only  one  (.898)  of  the  coefficients  differed 
significantly  from  zero.  However,  each  coefficient  is  based  on  only  5  pairs 
of  data.  The  pooled  correlation  coefficients  do  not  differ  significantly  from 
zero. 


The  reaction  time  data  for  the  second  study  were  also  used  to  test  for 
shape  (square  vs.  circle),  side  (right  vs.  left),  and  type  of  presentation  (4) 
differences.  A  repeated  measurement  (shape,  side,  type,  and  session)  analysis 
of  variance  procedure  was  used  to  test  for  differences  in  reaction  time.  Ses¬ 
sion  did  not  show  any  significant  interaction  with  any  of  the  other  three 
factors.  Table  9  lists  the  mean  reaction  time  for  the  various  types  of  mani¬ 
kin  presentations.  The  pattern  of  the  RT  means  for  shape  and  side  are  gener¬ 
ally  quite  comparable  for  the  different  presentation  levels.  The  types  of 
presentation  were  ranked  erect  away,  inverted  facing,  erect  facing,  and  in¬ 
verted  away  in  ascending  order  of  reaction  time  and  difficulty  (p=.001). 
Table  9  also  shows  the  effect  of  shape  and  side.  Discs  did  not  differ  from 
squares,  but  there  was  a  significant  (p<.001)  difference  of  right  from  left. 
The  right-sided  responses  were  approximately  25  msec  faster  than  the  left. 

Performance  with  emphasis  on  the  similarity  of  successive  presentations 
was  also  studied.  All  presentations  were  classified  by  degree  of  similarity 
from  identical  (1)  to  gross  dissimilarity  (7)  as  described  in  Table  10.  The 
mean  reaction  times  for  the  second  presentations  of  each  successive  pair  for 
these  sequences  are  listed  in  Table  10.  The  order  of  similarity  is  not  echoed 
exactly  in  the  mean  reaction  time  ranking,  but  a  similar  trend  is  seen.  The 
reaction  times  with  the  grossly  dissimilar  sequences  (similarity  6  and  7)  dif¬ 
fer  significantly  (p<.01)  from  the  moderately  dissimilar  sequence  reaction 
times.  In  order  to  equalize  the  variances  in  this  analysis,  a  log  transform 
of  each  RT  value  was  used  in  a  two-way  (proportional )  analysis  of  variance. 


TABLE  7.  CORRELATION  BETWEEN  OBJECTIVE  AND  SUBJECTIVE 
PERFORMANCE  RATINGS 


Mean  RT  across  all  sessions  (N= 1 7 ) 


RT  vs.  workload  RT  vs.  performance 


.028 


-.337 


TABLE  8.  SUBJECTIVE  VS.  OBJECTIVE  ASSESSMENTS 


Reaction  time  compared  with 

Subjects  Performance  Workload  Performance _ Workload 

Min  Max  Mi n  Max 


WS 

.048 

-.714 

4 

5 

3 

4 

SG 

-.595 

.786 

3 

6 

2 

4 

DB 

-.838 

.726 

3 

5 

1 

2 

LP 

-.189 

.758 

3 

6 

2 

3 

PD 

-.100 

.000 

3 

6 

4 

4 

DM 

-.488 

-.253 

3 

6 

2 

3 

BW 

-.075 

.000 

4 

5 

2 

2 

PP 

-.286 

.323 

3 

5 

2 

3 

EC 

.000 

.000 

4 

4 

2 

2 

FA 

-.434 

.463 

3 

5 

2 

3 

DT 

JB 

-.178 

Excl  tided 

.000 

3 

5 

2 

2 

RC 

-.594 

.787 

3 

5 

3 

4 

TB 

-.032 

.898 

2 

7 

2 

4 

VP 

.311 

-.809 

3 

5 

2 

3 

TW 

-.189 

.492 

1 

4 

2 

3 

JS 

-.221 

.000 

3 

7 

3 

3 

MF 

.270 

.000 

4 

6 

2 

2 

io  1  ed 

1 

• 

no 

-e* 

.264 
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TABLE  9.  MEAN  REACTION  TIME  FOR  VARIOUS  PRESENTATIONS 


Presentation 


Shape 

Side 

Erect 

away 

Erect 

facing 

Inverted 

facing 

Inverted 

away 

Total 

Circle 

Left 

.626 

.664 

.670 

.713 

.668 

Ri  ght 

.610 

.651 

.637 

.678 

.644 

Square 

Left 

.630 

.670 

.658 

.705 

.666 

Right 

.612 

.635 

.648 

.664 

.640 

Total 

.620 

.655 

.653 

.690 

TABLE  10. 

MEAN  REACTION  TIME  WITH  VARYING  DEGREES 

OF 

SIMILARITY  IN 

SEQUENCE 

Similarity 

Description 

N* 

Mean  RT 

Order 

1 

Same  presentation,  shape,  side 

19 

.615 

1 

2 

Same  presentation,  side 

39 

.636 

5 

3 

Same  presentation 

156 

.619 

4 

4 

All  erect,  same  side 

All  inverted,  same  side 

373 

.618 

3 

5 

All  erect 

All  inverted 

314 

.616 

2 

6 

All  different  but  same  side 

471 

.644 

6 

7 

All  different 

510 

.645 

7 

3,  4,  5  differ  significantly  from  6,  7.  ( p<  .01 ) 

♦Number  of  tasks/subjects. 
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A  major  criterion  of  any  psychomotor  task  is  that  it  should  produce  sta¬ 
ble  results  in  stable  conditions;  i.e.,  the  differential  stability  should  be 
high.  Bittner  (1)  has  suggested  that  all  tasks  should  be  examined  for  stabil¬ 
ity  and  proposed  a  graphic  analysis  strategy  for  this.  Pearson  product-moment 
correlation  coefficients  were  calculated  between  all  pair-wise  session  means 
(1-15)  using  the  reaction  data  with  only  correct  responses  for  the  17  sub¬ 
jects.  These  are  listed  in  Table  11.  The  lowest  between-sessions  correlation 
was  .56  (sessions  1-13),  indicating  a  fair  degree  of  stability  even  without 
extensive  practice.  A  selection  of  these  correlations  is  plotted  on  Figure  5 
for  sessions  1,  2,  4,  6,  10,  and  12.  Moreover,  the  manikin  task  may  be  seen 
to  have  good  task  reliability  with  an  estimate  of  the  common  correlation  of 
.84. 


TABLE  11.  CORRELATIONS  OF  MEAN  PERFORMANCE  WITH  TIME 


Session 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

1 

.82 

.73 

.70 

.77 

.71 

.68 

.60 

.74 

.65 

.62 

.59 

.56 

.65 

.66 

2 

.94 

.94 

.95 

.89 

.88 

.80 

.87 

.90 

.77 

.77 

.73 

.77 

.82 

3 

.98 

.98 

.94 

.89 

.86 

.87 

.92 

.69 

.72 

.67 

.72 

.85 

4 

.98 

.95 

.91 

.87 

.88 

.93 

.73 

.77 

.70 

.75 

.84 

5 

.95 

.91 

.87 

.92 

.92 

.72 

.76 

.71 

.74 

.87 

6 

.94 

.91 

.91 

.94 

.71 

.75 

.69 

.75 

.82 

7 

.97 

.97 

.97 

.85 

.85 

.78 

.84 

.86 

8 

.95 

.95 

.80 

.81 

.76 

.80 

.37 

9 

.93 

.80 

.81 

.74 

.79 

.85 

10 

.84 

.87 

.82 

.85 

.89 

11 

.97 

.92 

.94 

.81 

12 

.93 

.92 

.84 

13 

.95 

.90 

14 

15 


DISCUSSION 

The  choice  of  a  criterion  for  plateau  performance  was  based  on  acceptable 
tolerance  levels  for  control  sessions  for  individual  subjects.  That  all  hut 
one  subject  achieved  plateau  performance  in  the  confines  of  the  experimental 
exposures  was  gratifying.  The  strategy  adopted  by  one  subject  (JR)  was  inter¬ 
esting  and  unique.  He  reasoned  that  the  logic  of  the  manikin  microprocessor 
demanded  a  fixed  sequence,  albeit  with  a  random  start  position.  Having  de¬ 
cided  that,  he  sought  to  validate  his  hypothesis  by  memorizing  sequences  and 
hunting  those  sequences  in  later  exposures.  In  that  event  he  was  very  suc¬ 
cessful  .  He  demonstrated  memory  scans  of  up  to  30  sequences  where  the  reac¬ 
tion  time  was  zero  and  correct  responses  were  recorded.  Accordingly,  as  his 
memory  bank  grew,  he  achieved  better  performance  and  never  achieved  a  pla¬ 
teau.  As  he  was  not  dissuaded  from  that  strategy,  he  persisted;  thus  ins  data 
are  misleading  and  were  excluded  from  the  analysis. 
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No  subject  achieved  plateau  performance  with  less  than  4  sessions  of 
training,  but  all  had  reached  the  criterion  in  10.  The  mean  was  s.db,  and 
this  figure  gives  good  guidance  for  future  training  sessions.  The  data  failed 
to  suggest  that  frequency  of  training,  age,  or  occupational  level  affect,  the 
rate  of  achievement  of  plateau  ;>erformance  which  enhances  the  utility  of  the 
task.  Admittedly,  the  age  span  used  in  the  experiment  was  small,  but  if  was 
the  maximum  practical ,  given  the  available  subjects. 

Although  the  number  of  training  sessions  required  for  plateau  performance 
showed  no  appreciable  differences  with  the  groups  used,  the  overall  perform¬ 
ance  did  provide  some  suggestions  that  the  group  that  trained  on  alternate 
days  had  both  a  lower  mean  reaction  time  and  a  better  record  of  accuracy. 
However,  the  number  of  subjects  in  the  training  groups  was  small  (A)  and  these 
observed,  albeit  significant,  differences  were  due  more  to  individual  differ¬ 
ences  than  predictable  group  distinctions.  Furthermore ,  the  groupings  of  age 
and  occupational  level  were  not  so  distinguished,  and  these  data  were  recorded 
before  plateau  performance  had  been  achieved. 

The  analysis  showed  that  all  subjects  considered  the  +  vpes  if  manikin 
presentation  as  the  following  order  of  difficulty  as  graded  by  the  mean  reac¬ 
tion  tines  for  each:  erect  away,  erect  facing,  inverted  facing,  and  inverted 
away.  This  corresponds  to  previous  analyses  which  led  to  the  adoption  in  the 
manikin  software  of  the  even  distribution  of  all  possibil ities  of  presentation 
in  each  It;  presentations.  This  permits  analysis  of  performance  for  each  It 
presentation  segment,  if  need  be,  e.g.,  for  rapid  changes  in  performance  or 
consciousness.  However,  no  such  segmental  analyses  were  performed  m  this 
study.  The  order  of  difficulty  is  a  reflection  of  the  complexity  of  trie  men¬ 
tal  reorientation  required  to  match  the  manikin  to  that  of  the  subject  .  The 
upright  stance  was  clearly  easier  than  the  inverted,  the  facing  iwa v  be*  •  •••*' 
than  facing  towards.  the  order  jf  difficulty  was  preserved  across  **■■■■ 
ouency  of  training  groups,  with  four  minor  executions. 

In  the  second  experiment  a  similar  an '.lysis  ''howeH  a  ,r»  y  .  .n'  ■  at  i  on 

in  this  sequence ;  i.e.,  the  exchange  in  the  rank  mg  of  erect  facing  and  in¬ 
verted  facing.  experiment  1  data  were  from  subjects  who  were  learning  *  lie 
task ;  thorn*  fro  i  l.xper iinent  d  came  from  trained  personnel  .  The  dif  ference  in 
the  order  nay  indicate  some  changes  in  strategy  with  learning  hut  is  probably 
not.  w.eth  close  examination  given  the  small  differences  in  mean  reaction  time 
;n  the  rankings. 

The  second  experiment,  was  an  attempt  to  val  idate  the  subjective  rating 
cards  used  by  the  Crew  Performance  Branch  of  t  he  dTAJ  School  of  Aerospace  Med¬ 
icine.  However,  these  cards  are  of  more  utility  when  conditions  are  variable; 
e.g.,  subjects  becoming  exhausted,  workloads  varying  from  inactivity  to  over¬ 
load,  etc.  In  this  experiment  the  workload  presented  was  identical  each  time, 
and  performance  varied  little.  Some  subjects  reflected  this  in  their  mvari- 
at.e  assessments.  Accordingly,  little  correlation  was  expected  between  subjec¬ 
tive  and  objective  assessments  and  little  was  observed.  Some  subjects  showed 
better  correl  at  ions  than  others,  part  ic.ilarly  with  workload,  but  the  bet  f  or 
correlations  were  not  significantly  distributed  to  any  one  of  the  groupings. 
Ihis  part  of  the  experiment,  was  a  minor  goal  of  the  study  and  does  no*  reduce 
the  validity  of  the  concept  of  subjective  assessments  used  except  ,  a  <  .nrso, 
in  very  Stable  conditions,  is  in  this  experiment  . 
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Hunting  for  discs  rather  than  squares  in  the  task  presented  no  champ-,  in 
difficulty  as  almost  identical  mean  reaction  times  were  obtained  tor  mch. 
However,  there  was  a  considerable  difference  in  the  right-handed  r<-,po"v  . 
compared  to  left.  All  but  two  subjects  were  right-handed,  and  the  mean  in¬ 
ference  undoubtedly  reflects  this  handedness  bias. 

The  second  experiment  also  examined  the  alteration  of  reaction  time  by 
similar  previous  presentations.  The  memory  of  a  previous  manikin  problem 
solved  should  have  made  the  solution  of  a  similar  problem  easier  and  hence  re¬ 
duce  the  reaction  time.  These  analyses  confirmed  this  hypothesis  of  ’he  repe¬ 
tition  effect. 

The  analysis  showed  that  the  logical  similarity  ranking  did  nor  co*'r*>- 
spond  exactly  to  the  reaction  time  ranking.  Rank  2  which  presented,  r.he 
presentation  and  the  same  side  as  the  previous  manikin  had  much  lor; or  'v.u.- 
tion  times  than  those  where  side  and  shape  were  different  (3  and  5). 
when  all  three  variables  were  altered  (types  of  presentation,  side.:--  sm 
the  reaction  times  were  30  msec  longer. 

The  results  of  the  analysis  of  differential  stability  (Tahir  ; uni 
Fig.  5)  are  encouraging.  They  show  the  repeatabi 1 ity  of  the  task  ia'u  trom 
one  session  to  another.  Some  instability  due  to  learning  is  to  be  ec'eb; 
note  the  consistently  lower  correlations  obtained  with  session  *n,, 

lowest  correlation  observed  is  .56. 

The  plots  in  Figure  5  show  no  overall  trend,  up  or  down,  the  ;  urges1 
changes  occur  between  sessions  10  and  11.  These  changes  were  can'-ed  1  . 
weekend  intervening  for  all  subjects  between  experiments  1  arid  2  *  ‘  h ■ 

study . 


CONCLUSIONS 

1.  The  manikin  task  is  simple  and  reliable  to  use. 

2.  All  subjects  can  learn  the  task  and  acquire  plateau  performance  • 

3.  Acquisition  of  plateau  performance  is  achieved  in  approx ima! ri v  ■  si 
sions  of  practice;  the  extremes  range  from  4  to  10  sessions. 

4.  The  rate  of  acquisition  of  plateau  (lerformance  is  essentially  oid-pi-idi 
of  the  frequency  of  practice  or  occupational  level  or  age  of  the  sub  ‘  •  . 

5.  The  various  presentations  do  represent  varying  difficulties  v  .  1  /  ; 

but  this  variation  is  constant  for  all  subjects.  The  sequence  of  pt. •• 

ensures  that  all  16  possibilities  (and  difficulties)  are  posed  every 

6.  Right-handed  responses  were  quicker  than  left-handed  >-e\.  >■  >-  , 
squares  and  discs  produced  similar  responses. 

7.  There  was  poor  correlation  between  subjective  and  objective  « 

perf ormance ,  but  this  was  due  to  the  constant,  control  conditions  rr  >  rt 
than  t.he  inadequacies  of  the  performance  measures. 


;  * 


A  repetitive  effect  occurs  with  si ;  rl  I  jri  t  ies  of  <•?>•  :  ’  •  ’■  -n 

guence .  However,  the  -nun  i  k  in  tusk  has  a  fixed  so-;  .enc*-* .  ■’■■■re;  r.-,  Mm-, 
feet  is  common  for  all  subjects  and  sessions. 

9.  The  task  has  a  high  measure  of  differential  stability,  *  hot  •  *}  ir.- ,  cor’rl 
performance  is  essentially  constant  with  time. 
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